The present invention relates to a data multiplexing method, a data recorded medium, a data recording apparatus and a program describing a recording method, and, more particularly, to a data recorded medium for multimedia data, a data recording apparatus and a stream recording method all of the MPEG system or the like.
As schemes for recording and transferring a vast amount of picture and voice information as digital data, encoding systems, such as the MPEG (Moving Picture Experts Group) system, have been established and become international standards of encoding, such as the ISO/IEC 11172 and ISO/IEC 13818. Those systems are used as encoding systems for digital satellite broadcasting, DVD and so forth.
Due to the development on digital video cameras and the appearance of large-capacity recording media, such as DVD-RAM and DVD-RW, there are growing demands for processing of digitally recorded pictures and voices and studies and development on fast retrieval, analysis and edition of such digital data are underway.
The MPEG system is explained below as one example. ISO/IEC 13818 (hereinafter referred to as “MPEG-2”) includes ISO/IEC 13818-1 (hereinafter called “system standard”), ISO/IEC 13818-2 (hereinafter called “video standard”) and ISO/IEC 13818-3 (hereinafter called “audio standard”). A data stream of the video standard and a data stream of the audio standard are multiplexed according to the system standard. According to the video standard and audio standard, picture and voice information are structured according to their encoding units and are detected by specific start codes and synchronization data. The system standard can multiplex not only video and audio information but also a parameter for an encoder and information, such as a data arrangement in a recording medium. Other systems than the MPEG audio standard can also be used in audio encoding.
At the time of multiplexing the aforementioned data streams for transfer and recording, the system standard divides each of the streams that conforms to the video standard and the audio standard by an adequate length to yield data blocks, adds headers to the data blocks to produce packets and multiplexes the data streams packet by packet. A packet consists of a header and a payload. Recorded in the header are information for video and audio synchronization and flags indicating the characteristic of data. A data stream of video, audio or the like is recorded in the payload. Packets according to MPEG-2 include a pack, a PES (Packetized Elementary Stream) packet, and a TS (Transport Stream) packet. As a data stream is divided into packets of an adequate length before being multiplexed, recording, transfer and reproduction of data can be performed efficiently. Time information, namely an SCR (System Clock Reference) and PCR (Program Clock Reference), recorded in the header is used as a read time for a buffer by the decoder and as information for synchronization of clocks for the encoder and decoder. Time information, such as a PTS (Presentation Time Stamp) and DTS (Decoding Time Stamp), is used as information for synchronization of video playback and audio playback. Based on the playback time information, the decoder reproduces streams of video, audio and so forth in synchronism with each other.
While video data is encoded picture by picture, it is handled with a unit of motion compensation or a group of pictures (GOP) consisting of a plurality of pictures as a unit in executing edition or the like. Audio data is encoded audio access unit (AAU) by AAU. In general, video data and audio data are independently packetized and multiplexed. Edition of a multiplexed data stream (multiplexed stream) is carried out video object unit (VOBU) by VOBU which is a unit consisting of a series of packets from a packet including the head of a GOP to a packet including the head of a next GOP. Because a video stream and an audio stream are stored in respective buffers in case where the multiplexed stream is separated into the video stream and audio stream for reproduction on the decoder side, video packets and audio packets are multiplexed at timings suitable for the associated buffers.
According to the conventional data multiplexing method of this type, for a video packet and an audio packet which have the same playback time, the video packet is multiplexed first in a multiplexed stream, followed by multiplexing of the audio packet with some delay. At the head and end of a VOBU, therefore, a video packet and an audio packet which include data to be reproduced at the same time are included in separate VOBUs.
This is described below more specifically. According to the conventional multiplexed stream, as shown in FIG. 2, each video packet 22 and each audio packet 23 are multiplexed at timings suitable for the respective buffers, so that the video packet 22 and audio packet 23 which are to be reproduced at the same time are stored in separate VOBUs near VOBU boundaries, such as the head and end of a VOBU. When such a multiplexed stream 21 is edited VOBU by VOBU, the edited stream is reproduced with intermittent voices, not in a seamless manner.
In case where a multiplexed stream is divided at a VOBU boundary 24 and a preceding VOBU is to be deleted, for example, an audio frame 26 of interest is multiplexed at a position of an audio packet 23 included in the preceding VOBU. When the preceding VOBU is deleted, therefore, the audio frame that should be reproduced at the same time as a video frame 25 included in the succeeding VOBU is lost. This raises a problem of intermittent voices at the time the succeeding VOBU is reproduced. This occurs because time stamps of the video packet 22 and the audio packet 23 are not compared with each other at the time of multiplexing.
In short, when VOBU-by-VOBU edition, such as partial deletion or changing the reproducing order, is performed on a multiplexed stream which has been multiplexed according to the prior art method, part of video data or audio data that should be reproduced at the same time is lost, resulting in intermittent voices or the like. This disables the proper reproduction of edited data. To perform edition on this multiplexed stream in such a way as to guarantee seamless reproduction, it is necessary to perform remultiplexing after checking the playback time packet by packet, not VOBU-by-VOBU edition, which leads to a vast amount of data processing and requires a very large editing apparatus.